The finRes suite is organised along the data-science pipeline where preprocessing, including data collection and wrangling, plays a major role and is often reported by data-scientists to amount up to 80% of work-time. finRes adresses the issue in two complementary packages that work in conjuction with most of the finRes dataset packages.
On the one hand the pullit package provides tools for data collection from Bloomberg. It returns clean and tidy, ready-to-use, data objects for other packages further down the pipeline to work with. On the other hand the storethat package works in concert with fewISOs and GICS to help store the data retrieved for off-Bloomberg consumption in R.
Both pullit and storethat work in tandem with the BBGsymbols package. The latter plays a central role in finRes where it helps pullit in interacting with Bloomberg via the interface provided by the Rblpapi package (Armstrong, Eddelbuettel, and Laing 2018) and storethat in storing the data retrieved.
pullit together with storethat are the two workhorses of the finRes suite. Using carefully selected Bloomberg datafields from BBGsymbols in tandem with Armstrong, Eddelbuettel, and Laing (2018)’s Bloomberg interface pullit provides the R user with easy access to Bloomberg financial data for a number of financial instruments including, at the time of writting, equity and equity-like securities, funds with the category encompassing any money-managing entity, as well as futures series & term structure individual contracts.
For each instrument category, pullit allows data retrieval for a corresponding category specific set of data types. An active Bloomberg connection is required to retrieve Bloomberg financial data using pullit.
Equity data comes in three major categories in pullit, market, book and info. For a given corporation market data records stock market activity while book data records business activity in the form of financial statements commonly refered to as ‘books’ and info refers to qualitative information. See the ‘fields’ dataset in BBGsymbols for a detailed list of the Bloomberg datafields available.
Retrieve market data from Bloomberg for the Archer-Daniels-Midland (ADM US Equity), Kraft-Heinz (KHC US Equity) and XPO Logistics (XPO US Equity) corporations with:
For financial statements data BBGsymbols replicates the Bloomberg ‘financial analysis’ monitor (FA
Retrieve the corresponding data for the abovementioned corporations with:
‘info’ encompasses a range of contemporaneous qualitative information including, but not limited to, firm’s name, security type, exchange where the security trades, country of incorporation, etc.
Retrieve the corresponding data for the abovementioned corporations with:
Fund data only comes in two categories, market and info. For a given fund market data records stock market activity while info data records contemporaneous qualitative information that includes a wide range of fund characteristics. Market historical data as well as contemporaneous qualitative data not only allow for an indepth assessment of the fund performance but potentially also the mapping of the performance to fund characteristics. See the ‘fields’ dataset in BBGsymbols for a detailed list of the Bloomberg datafields available.
Retrieve market data from Bloomberg for the SPDR S&P 500 ETF Trust (SPY US Equity), the SPDR Gold Shares investment fund (GLD US Equity) and the iShares MSCI Emerging Markets ETF (EEM US Equity) with:
Retrieve the corresponding qualitative data for the abovementioned funds with:
Futures data comes in three major categories in pullit, market, CFTC and info. For a given futures series market data records futures market price activity while CFTC data records market positions. ‘info’ on the other hand records contemporaneous qualitative information for the corresponding futures series, including but not limited to series name, term structure length, contract size, contract unit, etc. See the ‘fields’ dataset in BBGsymbols for a detailed list of the Bloomberg datafields available.
pullit helps retrieving both term structure as well as aggregated market data for futures. Term structure data records market activity for individual futures term structure contracts while aggregated data records market activity measures that are aggregated over the whole term structure for a particular futures series.
Retrieve market data from Bloomberg for the five first term structure contracts on the Corn - #2-yellow (C A Comdty), Eurodollar (EDA Comdty) and S&P500 - e-mini (ESA Index) futures series where the futures chains are constructed by rolling on liquidity (with active contract: “A”) with no roll adjustment (none: “N”) using:
Retrieve the corresponding aggregated futures market data with:
pullit helps retrieving CFTC futures market position data from Bloomberg. The Commodity Futures Trading Commission (CFTC) publishes the Commitments of Traders (COT) reports to help the public understand market dynamics. Specifically, the COT reports provide a breakdown of each Tuesday’s open interest for futures and options on futures markets in which 20 or more traders hold positions equal to or above the reporting levels established by the CFTC. See the ‘fields’ and ’’ datasets in BBGsymbols for a details.
Retrieve the corresponding futures market position data with:
‘info’ encompasses a range of contemporaneous qualitative information on the underlying futures series including, but not limited to, name for the series’ underlying, trading exchange, term structure length, contract size, etc.
Retrieve the corresponding data for the abovementioned futures series with:
The functions above return objects that not only carry the retrieved financial data but also complementary information that can be accessed using the @ operator or more formaly using bespoke accessor methods. Historical data functions for example return objects that carry a ‘tickers’ dataframe that indicates the tickers for which some data have been found, a ‘fields’ dataframe that indicates the data fields for which data has been found, a ‘data’ dataframe that hosts the retrieved data as well as a character vector hosting the original call to the function. A get_periods() method complements the accessor methods by indicating the start and end dates between which data have been found for each ticker and data field:
futures_TS
#> S4 object of class FuturesTS
#>
#> Slots inlude
#> active_contract_tickers: access with get_active_contract_tickers()
#> term_structure_tickers: access with get_term_structure_tickers()
#> fields: access with get_fields()
#> data: access with get_data()
#> call: access with get_call()
#>
#> See also: get_periods()Access each slot using the appropriate accessor:
get_active_contract_tickers(futures_TS)
#> active contract ticker
#> 1: C A Comdty
#> 2: EDA Comdty
#> 3: ESA Indexget_fields(futures_TS)
#> active contract ticker ticker instrument book
#> 1: C A Comdty C 1 A:00_0_N Comdty futures market
#> 2: C A Comdty C 1 A:00_0_N Comdty futures market
#> 3: C A Comdty C 1 A:00_0_N Comdty futures market
#> 4: C A Comdty C 1 A:00_0_N Comdty futures market
#> 5: C A Comdty C 1 A:00_0_N Comdty futures market
#> ---
#> 129: ESA Index ES5 A:00_0_N Index futures market
#> 130: ESA Index ES5 A:00_0_N Index futures market
#> 131: ESA Index ES5 A:00_0_N Index futures market
#> 132: ESA Index ES5 A:00_0_N Index futures market
#> 133: ESA Index ES5 A:00_0_N Index futures market
#> type symbol
#> 1: term structure OPEN_INT
#> 2: term structure PX_ASK
#> 3: term structure PX_BID
#> 4: term structure PX_HIGH
#> 5: term structure PX_LAST
#> ---
#> 129: term structure PX_HIGH
#> 130: term structure PX_LAST
#> 131: term structure PX_LOW
#> 132: term structure PX_OPEN
#> 133: term structure PX_VOLUMEpullit::get_data(futures_TS)
#> ticker field date value
#> 1: C 1 A:00_0_N Comdty OPEN_INT 2015-11-18 557731
#> 2: C 1 A:00_0_N Comdty OPEN_INT 2015-11-19 577455
#> 3: C 1 A:00_0_N Comdty OPEN_INT 2015-11-20 588796
#> 4: C 1 A:00_0_N Comdty OPEN_INT 2015-11-23 613916
#> 5: C 1 A:00_0_N Comdty OPEN_INT 2015-11-24 646937
#> ---
#> 62933: ES5 A:00_0_N Index PX_VOLUME 2017-11-08 8
#> 62934: ES5 A:00_0_N Index PX_VOLUME 2017-11-09 1
#> 62935: ES5 A:00_0_N Index PX_VOLUME 2017-11-10 8
#> 62936: ES5 A:00_0_N Index PX_VOLUME 2017-11-15 32
#> 62937: ES5 A:00_0_N Index PX_VOLUME 2017-11-16 6pullit::get_call(futures_TS)
#> storethat_futures_TS(file = file, active_contract_tickers = active_contract_tickers,
#> start = start, end = end, TS_positions = TS_positions, roll_type = roll_type,
#> roll_days = roll_days, roll_months = roll_months, roll_adjustment = roll_adjustment,
#> verbose = verbose)get_periods(futures_TS)
#> active contract ticker ticker field start
#> 1: C A Comdty C 1 A:00_0_N Comdty OPEN_INT 2015-11-18
#> 2: C A Comdty C 1 A:00_0_N Comdty PX_ASK 2015-11-18
#> 3: C A Comdty C 1 A:00_0_N Comdty PX_BID 2015-11-18
#> 4: C A Comdty C 1 A:00_0_N Comdty PX_HIGH 2015-11-18
#> 5: C A Comdty C 1 A:00_0_N Comdty PX_LAST 2015-11-18
#> ---
#> 129: ESA Index ES5 A:00_0_N Index PX_HIGH 2015-11-18
#> 130: ESA Index ES5 A:00_0_N Index PX_LAST 2015-11-18
#> 131: ESA Index ES5 A:00_0_N Index PX_LOW 2015-11-18
#> 132: ESA Index ES5 A:00_0_N Index PX_OPEN 2015-11-18
#> 133: ESA Index ES5 A:00_0_N Index PX_VOLUME 2015-11-18
#> end
#> 1: 2017-11-17
#> 2: 2017-11-17
#> 3: 2017-11-17
#> 4: 2017-11-17
#> 5: 2017-11-17
#> ---
#> 129: 2017-11-15
#> 130: 2017-11-17
#> 131: 2017-11-15
#> 132: 2017-11-15
#> 133: 2017-11-16All the objects above can be stored in a bespoke database for later off-Bloomberg consumption. The storethat package makes the process seamless by providing befitted storing methods:
library(storethat)
db_create()
db_store(object = futures_TS, file = "~/storethat.sqlite", verbose = FALSE)
db_store(object = fund_market, file = "~/storethat.sqlite", verbose = FALSE)Every function above has an equivalent for retrieving data from a storethat database that can be accessed by swapping the ‘BBG’ prefix for ‘storethat’; i.e. ‘BBG_futures_market’ becomes ‘storethat_futures_market’. Function parameters are identical for both data sources and the returned objects have alike characteristics:
Updating a storethat database is equally straightforward with pullit. Update the equity content of the database with:
Narrower updates are also allowed:
The plotit package, also part of the finRes suite, provides plot methods for some pullit data objects including, at the time of writing, futures term structure (FuturesTS) and fund market (FundMarket) objects.
Plot a futures series term structure dynamics with:
Plot historical fund performance with:
Armstrong, Whit, Dirk Eddelbuettel, and John Laing. 2018. Rblpapi: R Interface to ’Bloomberg’. https://CRAN.R-project.org/package=Rblpapi.